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Abstract 

Cameras  for  automotive  use  need  to  provide  sufficient 
contrast  resolution  under  many  light  conditions.  The 
authors  have  developed  a  testing  procedure,  adapted 
from  detection  experiment  methodology >  used  for 
evaluating  vehicle  camouflage,  that  checks  a  camera's 
ability >  to  perform  in  situations  where  there  is  high 
light  level  in  some  areas  and  low  light  level  in  others. 
The  procedure  simulates  the  situation  of  looking  into 
oncoming  traffic  at  night  and  gives  a  quantitative 
measure  of  the  resolving  ability  of  the  camera  as  the 
lighting  is  varied. 

1.  Introduction 

The  use  of  cameras  in  automotive  applications  for 
safety,  comfort  and  convenience  is  increasing  rapidly. 
Applications  such  as  blind  spot  warning,  pedestrian 
recognition,  adaptive  cruise  control  and  driver 
recognition  all  require  cameras  that  work  under 
greatly  varying  light  conditions  [1,2].  Cameras  for 
automotive  use  need  to  provide  sufficient  contrast 
resolution  under  these  conditions. 

The  most  challenging  light  condition  is  when  a 
portion  of  the  scene  is  brightly  lit,  and  another  portion 
is  in  deep  shadow.  This  situation  can  occur  whenever 
the  sun  is  in  the  field  of  view.  In  northern  locales  this 
occurs  most  of  the  day  in  winter.  At  night  when  other 
vehicles  are  present,  this  is  a  very  common  situation 
when  another  vehicle's  headlights  are  within  the  field 
of  view. 

Camera  specifications  from  the  manufacturer  typically 
contain  information  such  as  lowest  operable  light  level 
and  resolution.  However  this  information  is  not 
usually  sufficient  to  characterize  a  camera  for 
automotive  applications.  Most  low  light  cameras 
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perform  well  when  the  entire  scene  is  dark,  but 
provide  almost  no  contrast  if  there  is  a  bright  object  in 
the  scene.  Even  if  the  camera  doesn't  bloom,  this  lack 
of  contrast  makes  the  cameras  unsuitable  for 
automotive  use. 

2.Experimental  Procedure 

The  authors  have  developed  a  testing  procedure  that 
checks  a  camera's  ability  to  perform  in  situations 
where  there  is  high  light  level  in  some  areas  and  low 
light  level  in  others.  The  procedure  simulates  the 
situation  of  looking  into  oncoming  traffic  at  night. 

The  test  procedure  is  adapted  from  detection 
experiment  methodology  used  for  evaluating  vehicle 
camouflage  and  gives  a  quantitative  measure  of  the 
resolving  ability  of  the  camera  as  the  lighting  is 
varied.  The  camera  under  test  is  focused  on  a  USAF 
Tribar  of  a  size  that  is  easily  resolvable  in  normal 
lighting.  A  standard  60-watt  light  bulb  is  placed  in 
front  of  a  standard  size  white  sheet  of  paper  half  way 
between  the  target  and  the  camera.  Fig.  1  and  2  show 
the  experimental  set-up  for  this  test.  The  distance  D 
(see  Fig.  1)  from  the  camera  to  the  target  is 
determined  by  observing  the  image  on  a  monitor  and 
adjusting  the  distance  until  the  target  is  %  the  height 
of  the  screen  and  its  center  is  1/3  of  the  screen’s  width 
from  the  right  hand  edge  (see  Fig.  2).  This  method 
was  chosen  to  make  the  scene  viewed  by  each  camera 
similar  regardless  of  the  camera’s  field  of  view. 

Camera  performance  under  varying  light  conditions 
was  assessed  by  progressively  blocking  off  the  image 
of  the  bulb,  as  seen  by  the  camera,  and  recording  the 
response  from  4  observers  as  to  how  detectable  the 
target  appeared  on  a  monitor.  The  detectability  levels 
were  as  follows: 

0.  can  see  nothing 

1 .  can  tell  something  is  there 

2.  can  resolve  2  separate  groups  of  something 
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3.  can  clearly  resolve  one  group  of  3  bars 

4.  can  clearly  resolve  both  the  horizontal  and  vertical 
bars. 


The  test  began  with  the  light  bulb  fully  exposed  to  the 
camera.  The  camera’s  view  of  the  light  bulb  was  then 
progressively  blocked  with  a  black  shield.  The 
observers  were  asked  to  assess  the  detectability  of  the 
target  for  20,  40,  60,  80  and  100  %  blocking  of  the 
bulb.  Light  levels  at  the  camera  and  the  targets  were 
measured  for  each  test  with  a  Photo  Research 
Spectroradiometer. 
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Fig.  1.  Experimental  Setup  to  measure  detectability 
of  AF  3-  bar  targets. 


Glare  Test  Screen  View 


Fig.  2:  Position  of  bar  target  relative  to  glare 
source 


Fig.  3:  Position  of  source  relative  to  target  in 
camera  FOV 


An  ANalysis  Of  VAriance  (ANOVA)  was  then  done 
to  determine  if  there  were  significant  differences 
between  the  cameras  tested.  The  results  of  this 
analysis  are  detailed  in  the  next  sections. 

A  second  series  of  tests  were  performed  on  three  of 
the  original  test  cameras  using  two  resolution  targets 
illuminated  at  different  light  levels.  Varying  the 
incident  light  level  in  detectability  increments  created 
a  detailed  characterization  curve  of  these  cameras.  The 
light  level  was  measured  at  the  target  with  a 
photometer.  Fig.  4  shows  the  experimental  setup.  The 
targets  were  displayed  in  “cubby  holes”  one  meter  on 
a  side  which  allowed  dramatically  different  light 
levels  to  be  used  on  the  targets.  The  targets  were 
illuminated  by  150-watt  spotlights  whose  brightness 
could  be  changed  by  the  use  of  individual  variable 
transformers.  The  distance  from  the  tested  camera  to 
the  target  was  adjusted  to  achieve  the  scene  shown  on 
the  monitor  in  Fig.  5.  The  tests  began  with  both 
targets  illuminated  at  a  level  just  sufficient  to  allow 
resolution  of  both  targets.  The  light  level  on  the  left 
target  was  held  constant  during  the  experiment  while 
the  light  on  the  right  target  was  increased  until  the 
resolution  degraded.  Progressive  degradation  as  the 
light  level  at  the  target  was  increased  was  reported  by 
the  4  observers  and  recorded  using  the  same  0  to  4 
scale  used  in  the  previous  experiments.  The  light 
levels  at  that  target  were  recorded  when  a  transition  in 
detectability  was  reported  by  a  majority  of  observers. 
This  test  yielded  a  relationship  between  detectability 
and  light  level  as  the  illumination  on  the  target  is 
increased  beyond  the  optimum  level.  This  relationship 
is  important  for  automotive  applications  since  it  is 
rarely  possible  to  achieve  any  control  over  scene 
lighting  much  less  achieve  optimal  conditions.  Table  7 
summarizes  the  data  from  these  tests.  Figure  13  shows 
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a  representative  spectrum  of  the  light  incident  on  the 
target. 


Fig.4:  Setup  used  to  measure  target  luminance 


Since  the  detectability  of  the  bar-targets  was  measured 
on  an  ordinal  scale,  a  rank  transformation  was 
performed  on  the  original  data.  Fig.  6  represents  the 
box-and-whisker  plots  for  the  rank  of  detectability  for 
the  five  camera  types.  The  following  analysis  of 
variance  was  performed  on  the  transformed  data. 

Table  2  below  shows  the  design  of  the  experiment  to 
rank  the  cameras.  The  authors  used  five  cameras,  and 
five  shield  positions,  and  four  observers.  Once  a 
camera  is  chosen,  the  subject  and  the  five  levels  of 
position  are  randomly  determined.  Thus,  we  have  a 
randomized  complete  block  experiment  run  in  a 
randomized  complete  block. 


The  linear  mathematical  model  for  this  experiment  is 
given  by  Equation  (1). 

The  cameras  used  in  both  parts  of  this  experiment  are 
shown  below  in  Table  1.  The  Sony  camera  was  used 
with  and  without  a  6  mm  lens. 

yjk  =  M+Ti+ Pj + sk  +  £ijk  < 


Where,  ii  is  the  shield  position  effect,  ft,  represents 
the  block  effect  due  to  the  subject,  8k  is  the  block 
effect  due  to  the  camera,  and  Sjjk  is  the  normally 
distributed  error  tenn. 


The  complete  analysis  of  variance  is  summarized  in 
Table  3.  Both  CAMERA  and  POSITION  are  highly 
significant.  The  type  of  camera  used,  therefore,  does 
effect  the  subject’s  ability  to  detect  the  target.  A 
normal  probability  plot  of  the  standardized  residuals  is 
shown  in  Fig.  7.  The  assumption  that  the  residuals  are 
normally  distributed  is  verified.  Figs.  8  through  1 1 
show  the  standardized  residuals  plotted  against  the 
fitted  values  and  independent  variables.  These  plots 
reveal  that  the  assumption  of  constant  variance  is 
satisfied. 


Fig.  5:  Sample  image  showing  gain  problem 


Table  1 

Video  Cameras  Tested 


Camera 

Name 

Picture 

elements 

Min.  Illumination 

ELMO 

QN42H 

786X494 

20  lux 

Panasonic 

GPKS162 

768  X  494 

3 

Sony  DC50A 

768  X  494 

0.8 

Genwac  GW- 
902H 

768  X  494 

0.0003 

i  =  1,2, 3, 4, 5 
7  =1,2,3, 4  (1) 

*  =  1,2,3, 4,5. 


3.  Data  Analysis 

From  the  observer  data  on  the  detectability  of  the  five 
cameras,  an  Analysis  of  Variance  was  done  to 
determine  the  camera  with  the  highest  gain  quality 
and  best  contrast. 
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Table  2 

Between  Subjects  Factors 


Factor 

Label 

Value 

N 

Camera 

1.0 

Sony 

20 

Type 

2.0 

Elmo 

20 

3.0 

Panasonic 

20 

5.0 

Gemwac 

20 

6.0 

Sony- 

6mm 

20 

Position 

0.0 

0 

20 

Of 

5.0 

5 

20 

Shield 

10.0 

10 

20 

15.0 

15 

20 

20.0 

20 

20 

Subject 

1 

20 

2 

20 

3 

20 

4 

20 

Table  3 

Between  Subject  Effects 


Dependent  Variable:  RANK  of  DETECT 


Source 

Type  III 
Sum  of 
Squares 

df 

Mean 

Square 

F 

Sig. 

Noncent. 

Parameter 

Observed 

Power3 

Corrected 

Model 

>0034.390b 

11 

5457.672 

25.237 

.000 

277.607 

1.000 

Intercept 

255025.0 

1 

255025.0 

1179.269 

.000 

1179.269 

1.000 

CAMERA 

5658.325 

4 

1414.581 

6.541 

.000 

26.165 

.989 

POSITION 

>4326.025 

4 

13581.506 

62.803 

.000 

251.211 

1.000 

SUBJECT 

50.040 

3 

16.680 

.077 

.972 

.231 

.063 

Error 

Total 

Corrected 

Total 

9030.610 

334090.0 

'9065.000 

88 

100 

99 

216.257 

a-  Computed  using  alpha  =  .05 

b-  R  Squared  =  .759  (Adjusted  R  Squared  =  .729) 


Estimated  Mean  of  Rank  of  Camera  Type 


Dependent  Variable:  RANK  of  DETECT 


camera  type 

Mean 

Std.  Error 

Sony 

61.50000 

3.288 

Elmo 

45.90000 

3.288 

Panasonic 

55.12500 

3.288 

Gemwac-6mm 

50.40000 

3.288 

Sony-6mm 

39.57500 

3.288 

Table  5 

Multiple  Comparisons 


Dependent  Variable:  RANK  of  DETECT 
Tukey  HSD _ 


(1)  camera  type 

(J)  camera 
type 

Mean 

Difference 

(i-j) 

Std.  Error 

Sig. 

95%  Confidence 
Interval 

Lower 

Bound 

Upper 

Bound 

Sony 

Elmo 

15.60000* 

4.650 

.010 

2.64799 

28.55201 

Panasonic 

6.37500 

4.650 

.648 

-6.57701 

19.32701 

Gemwac-6mm 

11.10000 

4.650 

.129 

-1.85201 

24.05201 

Sony-6mm 

21.92500* 

4.650 

.000 

8.97299 

34.87701 

Elmo 

Sony 

-15.60000* 

4.650 

.010 

-28.55201 

-2.64799 

Panasonic 

-9.22500 

4.650 

.282 

-22.17701 

3.72701 

Gemwac-6mm 

-4.50000 

4.650 

.869 

-17.45201 

8.45201 

Sony-6mm 

6.32500 

4.650 

.655 

-6.62701 

19.27701 

Panasonic 

Sony 

-6.37500 

4.650 

.648 

-19.32701 

6.57701 

Elmo 

9.22500 

4.650 

.282 

-3.72701 

22.17701 

Gemwac-6mm 

4.72500 

4.650 

.847 

-8.22701 

17.67701 

Sony-6mm 

15.55000* 

4.650 

.010 

2.59799 

28.50201 

Gemwac-6mm 

Sony 

-11.10000 

4.650 

.129 

-24.05201 

1.85201 

Elmo 

4.50000 

4.650 

.869 

-8.45201 

17.45201 

Panasonic 

-4.72500 

4.650 

.847 

-17.67701 

8.22701 

Sony-6mm 

10.82500 

4.650 

.146 

-2.12701 

23.77701 

Sony-6mm 

Sony 

-21.92500* 

4.650 

.000 

-34.87701 

-8.97299 

Elmo 

-6.32500 

4.650 

.655 

-19.27701 

6.62701 

Panasonic 

-15.55000* 

4.650 

.010 

-28.50201 

-2.59799 

Gemwac-6mm 

-10.82500 

4.650 

.146 

-23.77701 

2.12701 

Based  on  observed  means.  The  error  term  is  Error. 

*•  The  mean  difference  is  significant  at  the  .05  level. 


Table  6 

Homogenous  Subsets  of  Detectability  Rank 


Table  3  shows  the  significance  of  the  various  factors. 
Subject  turned  out  not  to  be  a  statistically  significant 
variable.  Table  4  below  shows  how  the  cameras 
ranked  relative  to  each  other  based  on  the  transformed 
test  means  and  Table  5  shows  the  relative  significance 
of  each  factor  and  the  standard  errors,  and  Table  6 
shows  the  subsets  of  cameras  that  were  not 
statistically  different. 


RANK  of  DETECT 


Tukey  HSC?’ b 


camera  type 

N 

Subset 

i 

2 

3 

Sony-6mm 

20 

39.57500 

Elmo 

20 

45.90000 

45.90000 

Gemwac-6mm 

20 

50.40000 

50.40000 

50.40000 

Panasonic 

20 

55.12500 

55.12500 

Sony 

20 

61.50000 

Sig. 

.146 

.282 

.129 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  216.257. 


a-  Uses  Harmonic  Mean  Sample  Size  =  20.000. 
b.  Alpha  =  .05. 

Based  on  “Tukey’ s  honesty  significant  difference 
test”,  the  starred  values  in  Table  5  indicate  the  pair  of 
means  that  are  significantly  different; 

C1-C2* 


Table  4 
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C1-C3 

C1-C5 

C1-C6* 

C2-C3 

C2-C5 

C2-C6 

C3-C5 

C3-C6* 

C5-C6 

Referring  to  Table  2,  Cl  is  the  Sony,  C2  is  the  Elmo, 
C3  is  the  Panasonic,  C5  is  the  Gemwac-6mm,  and  C6 
is  the  Sony-6mm.  Based  on  Tukey’s  test,  the  authors 
conclude  that  there  is  not  a  statistically  significant 
difference  in  image  quality  between  Cl,  C3,  and  C5. 
These  cameras  are  the  best  perfonning  of  the  five 
tested. 

120 

100 


when  compared  to  the  Sony-6mm  camera  and  not  a 
significant  difference  when  the  Elmo  camera  is 
compared  to  Sony-6mm  camera. 

Normal  P-P  Plot  of  Standardized  Residi 


Observed  Cum  Prob 

Figure  7:  Normal  Probability  Plot 
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Fig.  6  Box-and-whisker  plots  for  camera  types 

Fig.  12  is  a  graph  of  the  data  from  our  second  series  of 
experiments  and  shows  the  loss  of  detectability  with 
light  increasing  on  the  primary  target.  These  curves 
are  as  expected  from  our  original  experiments.  The 
left  two  curves  represent  the  Sony-6mm  camera  at  two 
slightly  different  initial  values  of  lighting.  They  are 
essentially  parallel  and  are  extremely  steep.  This 
confirms  the  perception  that  near  the  limits  of  zero 
illuminance  detectability  drops  off  rapidly.  The  slope 
of  the  curve  is  directly  related  to  the  dynamic  range  in 
this  region  with  steeper  slopes  indicating  less  dynamic 
range.  The  center  two  curves  are  for  the  Elmo  camera. 
Again  they  are  for  two  slightly  different  light  levels 
and  are  parallel  and  extremely  steep  with  essentially 
the  same  shape  as  the  curves  for  the  Sony  camera.  The 
rightmost  curve  is  for  the  Panasonic  camera  and  is 
distinctly  different  in  shape  from  the  other  curves.  The 
distinctly  different  shape  supports  the  conclusion  from 
the  ANOVA  multiple  comparison  in  Table  5  of  the 
first  set  of  experiments,  which  found  a  significant 
difference  in  the  data  from  the  Panasonic  camera 


Predicted  Value  for  RDETECT 

Fig.  8:  Standardized  Residuals  vs.  Predicted  Value 
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Fig.  9:  Standardized  Residuals  vs.  Camera  Type 
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Fig.  13:  Spectrum  of  Incident  Light  on  Target 


4.  Conclusions 
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Fig.  10:  Standardized  Residuals  vs.  Shield  Position 
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Fig.  11:  Standardized  Residuals  vs.  Subject 


Loss  Of  Detectability  With  Light  Increasing  On  Primary  Target 


Log  Light  Level 


~ Sony  0. 1 32_ Sony  1 .83 —^Panasonic  2.793 — Elmo  0.934_ Elmo  2.839 


The  authors  have  described  an  experimental  procedure 
to  characterize  the  gain  ability  of  video  cameras  for 
use  in  automotive  and  military  conditions  The  Sony 
camera  was  found  to  have  the  highest  ranking  in  the 
detectability  test.  The  method  the  authors  describe 
can  determine  which  camera  will  perform  the  best 
under  illumination  conditions  that  span  several  orders 
of  magnitude.  Future  work  will  involve  using  new 
“high-dynamic  range”  cameras  as  well  as  measuring 
the  time-response  of  the  cameras,  as  it  takes  a  finite 
time  for  the  cameras  to  adjust  after  the  introduction  of 
a  new  “high-glare”  source. 
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Fig  12:  Detectability  Loss  with  Increasing  Light  on 
Primary  Target 


Appendix 
Table  7 

Camera  data  from  Experiment  2 


camera  type 

sony-6mm 

Panasonic 

Elmo 

LHS  luminance  (cd/m2) 

0.132 

1.83 

2.793 

0.9336 

2.839 

log  (lum) 

-0.879426 

0.262451 1 

0.4460709 

-0.029839157 

0.453165393 

log(LUM) 

avg  resp 

log(LUM) 

avg  resp 

log(LUM) 

avg  resp 

log(LUM) 

avg  resp 

log(LUM) 

avg  resp 

Initial 

-0.9578 

4.00 

0.2428 

4.00 

0.4150 

3.75 

-0.0060 

4.00 

0.4853 

4.00 

A 

0.8349 

4.00 

0.7922 

4.00 

2.0913 

3.75 

1.7508 

3.50 

1 .7874 

3.75 

B 

0.8935 

3.50 

0.8439 

2.75 

2.1443 

3.00 

1.7550 

2.50 

1 .8004 

2.75 

C 

0.9182 

2.00 

0.8624 

1.75 

2.5113 

2.00 

1.7748 

1.50 

1.8321 

1.75 

D 

0.9492 

1.00 

0.8993 

1.00 

2.5412 

1.00 

1.7858 

0.75 

1 .8675 

1.00 

E 

0.9899 

0.25 

0.9388 

0.00 

2.5707 

0.25 

1.8309 

0.00 

1 .8963 

0.00 
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